Go to main content
5/32

# Changes in This Release for Oracle Big Data Appliance

This preface contains:

- Changes in Oracle Big Data Appliance Release 4 (4.4)

- Changes in Oracle Big Data Appliance Release 4 (4.3)

- Changes in Oracle Big Data Appliance Release 4 (4.2)

- Changes in Oracle Big Data Appliance Release 4 (4.1)

- Changes in Oracle Big Data Appliance Release 4 (4.0)

## Changes in Oracle Big Data Appliance Release 4 (4.4)

The following are changes in Oracle Big Data Appliance Release 4 (4.4):

Software Updates

- CDH (Cloudera's Distribution including Apache Hadoop) 5.5.1

- CDM (Cloudera Manager) 5.5.1

- Cloudera Navigator 2.4.1

- MySQL Database Enterprise Server - Advanced Edition 5.6

- Oracle Big Data Connectors 4.4

- Oracle Data Integrator Agent 12.2.1 (for Oracle Big Data Connectors)

- Oracle NoSQL Database 3.5.2

- Perfect Balance 2.6

Hardware Updates

- Oracle Big Data Appliance X6-2 server

  The X6-2 provides substantial increases in processing power and memory over X5–2 servers:

  - 2 x 22-core (2.2GHz) Intel® Xeon® E5-2699 v4 processors.

  - 8 x 32 GB DDR4-2400 memory (expandable to maximum of 768 GB per node).

  X6-2 servers are shipped with an Oracle Big Data Appliance v4.4.0 base image.

  In Oracle Big Data Appliance Release 4.4 or greater, X6-2 nodes can be mixed with X5-2 nodes (and older Release 4.4–compatible nodes) in a CDH or NoSQL cluster. The X6-2 server is not compatible as a node of an Oracle Big Data Appliance cluster in releases previous to 4.4.

  See the Oracle Big Data Appliance X6-2 Data Sheet for more details.

New Features

- Cloudera CDH 5.5.1 and Cloudera Manager 5.5.1

  CDH 5.5.1 is a maintenance release on top of CDH 5.5. See the Cloudera CDH 5.5 Release Notes

  For information on Cloudera Manager 5.5 and 5.5.1, see New Features and Changes in Cloudera Manager 5

- Automated Installation for Cloudera Navigator

  Mammoth now provides an automated installation for Cloudera Navigator in both a full Mammoth installation and Mammoth upgrade. No user intervention is required and the installation occurs transparently. If Cloudera Navigator is not already installed, Mammoth installs the software on node 3 of the cluster, which is where other Cloudera Management services are hosted. If Cloudera Navigator is already installed, Mammoth skips this step and does not overwrite the existing installation.

  The Cloudera Navigator Metadata Server and Audit Server are automatically added to Cloudera Manager and auditing is enabled. Mammoth also enables Web UI encryption for the Audit Server.

  Mammoth does not enable the Cloudera Navigator key management components.

- Support for Oracle Big Data SQL 3.0

  Oracle Big Data Appliance Release 4.4 includes Oracle Big Data SQL 2.0 as a Mammoth installation option. Oracle Big Data SQL 3.0 is also available for Release 4.4, as a patch. See the Oracle Big Data SQL User's Guide for Release 3.0 installation instructions.

  Note:

  If you want to install Oracle Big Data SQL 3.0, do not select Oracle Big Data SQL 2.0 in the Mammoth installation. If Oracle Big Data SQL 2.0 is installed, you must uninstall it prior to installing the 3.0 patch. The patch README file includes steps for removing 2.0 if you have previously installed it.

Release 4.4 as an Update to a Earlier Base Image

Mammoth 4.4.0 can run on top of any earlier Oracle Big Data Appliance 4.x base image and will update the base image software as needed.

## Changes in Oracle Big Data Appliance Release 4 (4.3)

The following are changes in Oracle Big Data Appliance release 4 (4.3):

Software Updates

- CDH (Cloudera's Distribution including Apache Hadoop) 5.4.7

- CDM (Cloudera Manager) 5.4.7

- Oracle Big Data Connectors 4.3

- Oracle Big Data Discovery 1.1.1

- Oracle Big Data SQL 2.0

- Oracle NoSQL Database 1.3.4.7 (Community and Enterprise Edition)

- Oracle Table Access for Hadoop and Spark

- Perfect Balance 2.5

- JDK 8u60

See Oracle Big Data Appliance Software User's Guide.

New Features

- Automatic Installation for Oracle Big Data Discovery

  Customers can download Big Data Discovery 1.1.1 and then use the bdacli command line utility to install the software on a designated node of the primary CDH cluster.

  See Expanding an Oracle Big Data Appliance Starter Rack.

- Oracle Table Access for Hadoop and Spark

  Oracle Table Access for Hadoop and Spark is an Oracle Big Data Appliance feature that converts Oracle Database tables into Hadoop or Spark data sources. This feature enables fast and secure access to data in the Oracle Database.

- HDFS Transparent Encryption

  Oracle Big Data Appliance 4.3 provides the option to use HDFS Transparent Encryption. This replaces the eCryptfs on-disk encryption software provided with previous releases. Customers can enable HDFS Transparent Encryption for both new and pre-existing CDH clusters. When enabled, HDFS Transparent Encryption secures Hadoop operations running on the cluster (including HDFS, MapReduce on YARN, Spark on YARN, Hive, and Hbase tasks).

  - The Oracle Big Data Appliance Configuration Generation Utility provides an option to include HDFS Transparent Encryption when a new cluster is created.

  - HDFS Transparent Encryption can be enabled or disabled on a cluster via the bdacli command line interface.

- HTTPS / Network Encryption

  - Provides HTTPS for Cloudera Manager, Hue, Oozie, and Hadoop Web UIs.

  - Enables network encryption for other internal Hadoop data transfers, such as those made through YARN shuffle and RPC.

  Like HDFS Transparent Encryption, HTTPS/ Network Encryption is an option in the Oracle Big Data Appliance Configuration Generation Utility, and can also be enabled via bdacli.

- Zero Downtime for Upgrades, One-Off Patches, and Cluster Extensions

  In Release 4.3, Oracle Big Data Appliance leverages Cloudera’s Rolling Upgrades functionality to keep clusters operational during Mammoth upgrades, patches, and cluster extensions. This is an installation option that allows certain services on a cluster to remain continuously available while each node in the cluster is upgraded and rebooted. Zero Downtime is an option for the following tasks:

  - Upgrades of the Mammoth software (including Cloudera's Distribution Including Apache Hadoop, Cloudera Manager, and the Mammoth software itself).

  - One-off patches of Mammoth-installed software.

  - Cluster extensions. (For cluster extensions within a single rack, rolling upgrades are not optional. These extensions are always done as rolling upgrades.)

Deprecated Features

The following features are deprecated in this release, and may be desupported in a future release:

- MapReduce 1 (MRv1)

  YARN (MRv2) supersedes MRv1. Users who want to continue to use MRv1 on Oracle Big Data Appliance versions 3.x and 4.x should contact Oracle Support before using Mammoth to patch or upgrade the software.

Desupported Features

The following features are no longer supported as of this release:

- eCryptfs On-Disk Encryption

  This has been replaced by HDFS Transparent Encryption.

## Changes in Oracle Big Data Appliance Release 4 (4.2)

The following are changes in Oracle Big Data Appliance release 4 (4.2):

New Features

- Software Upgrades

  - Cloudera's Distribution including Apache Hadoop 5.4.0

  - Cloudera Manager 5.4.0

  - Perfect Balance 2.4.0

  - Oracle Big Data SQL 1.1

  - No SQL Database 3.2.5

  - Oracle Linux 6.6 and 5.11

  - JDK 8u45

  See Oracle Big Data Appliance Software User's Guide.

- Hardware Upgrades

  - Oracle Big Data Appliance is now shipped with 8 TB disk drives

- Elastic Configuration

  - Oracle Big Data Appliance now provides the flexibility of adding one or more servers on a starter rack using Big Data Appliance X5-2 High Capacity Nodes plus InfiniBand Infrastructure. You can add up to 12 additional servers on a starter rack.

    See Expanding an Oracle Big Data Appliance Starter Rack.

- Automatic Installation Support

  - Spark-on-YARN is deployed automatically

  - Oracle Spatial and Graph is installed and configured automatically

- Oracle Big Data SQL 1.1

  - Copy to BDA

    This utility enables you to copy relatively static tables from an Oracle database into Hadoop, with the purpose of improving query times.

    See Oracle Big Data Appliance Software User's Guide.

  - Oracle NoSQL Database Support

    Oracle databases on Oracle Exadata Database Machine can use Oracle Big Data SQL to connect to clusters running Oracle NoSQL Database.

  - Parquet Support

    CDH 5.2 and later versions include Hive 0.13, which supports the Apache Parquet file format. This file format is used by Cloudera Impala and other Hadoop software.

    See Oracle Big Data Appliance Software User's Guide.

Other Changes

- Oracle Big Data Appliance X5-2

  Oracle Big Data Appliance 4.2 software supports Oracle Big Data Appliance X5-2 and earlier version server hardware.

  See "Server Components".

- Oracle Big Data Appliance Configuration Generation Utility

  This utility generates two new configuration files:

  - `network.json`: Supersedes `BdaDeploy.json`. For software upgrades, Mammoth converts the existing`BdaDeploy.json` to `network.json`. New installations must have `network.json`.

  - `networkexpansion.json`: Supersedes `BdaExpansion.json`.

  See "About the Configuration Files".

- CDH Deployment

  Mammoth uses parcels instead of RPMs to deploy CDH.

- Apache Sentry

  Installation of Apache Sentry does not require `sentry-provider.ini` as a prerequisite.

- Microsoft Active Directory Server in Mammoth

  Support for directly using Microsoft Active Directory named as Active Directory Kerberos in Mammoth.

- Oracle Linux Support

  Oracle Linux 5 support for Oracle Big Data Appliance X5-2 servers.

- Cloudera Navigator Trustee Server

  Cloudera Navigator Trustee Server installer package and documentation are now shipped in Mammoth. It must be manually installed on a separate server.

Deprecated Features

The following features are deprecated in this release, and may be desupported in a future release:

- Mammoth Reconfiguration Utility

  The `bdacli` utility supersedes `mammoth-reconfig`. The `mammoth-reconfig` utility is only needed to change the disk encryption password.

  See "bdacli".

- MapReduce 1 (MRv1)

  YARN (MRv2) supersedes MRv1. Users who want to continue to use MRv1 on Oracle Big Data Appliance versions 3.x and 4.x should contact Oracle Support before using Mammoth to patch or upgrade the software.

- Disk Encryption

  A new encryption system that is more flexible and robust will replace the current system in an upcoming release.

## Changes in Oracle Big Data Appliance Release 4 (4.1)

The following are changes in Oracle Big Data Appliance release 4 (4.1):

New Features

- Software Upgrades

  - Cloudera's Distribution including Apache Hadoop 5.3.0

  - Cloudera Manager 5.3.0

  - Perfect Balance 2.3.0

  - Oracle Big Data SQL 1.1

  - Oracle Big Data Connectors 4.1

  - Oracle Linux 6.5

  See Oracle Big Data Appliance Software User's Guide.

- Oracle Big Data SQL 1.1

  - Copy to BDA

    This utility enables you to copy relatively static tables from an Oracle database into Hadoop, with the purpose of improving query times.

    See Oracle Big Data Appliance Software User's Guide.

  - Oracle NoSQL Database Support

    Oracle databases on Oracle Exadata Database Machine can use Oracle Big Data SQL to connect to clusters running Oracle NoSQL Database.

  - Parquet Support

    CDH 5.2 and later versions include Hive 0.13, which supports the Apache Parquet file format. This file format is used by Cloudera Impala and other Hadoop software.

    See Oracle Big Data Appliance Software User's Guide.

- Oracle NoSQL Database

  The `bdacli admin_cluster` command supports Oracle NoSQL Database nodes that require repair or replacement.

  See Oracle Big Data Appliance Software User's Guide.

Other Changes

- Oracle Big Data Appliance X5-2

  Oracle Big Data Appliance 4.1 software supports the Oracle Big Data Appliance X5-2 server hardware.

  See "Server Components".

- Oracle Big Data Appliance Configuration Generation Utility

  This utility generates two new configuration files:

  - `network.json`: Supersedes `BdaDeploy.json`. For software upgrades, Mammoth converts the existing`BdaDeploy.json` to `network.json`. New installations must have `network.json`.

  - `networkexpansion.json`: Supersedes `BdaExpansion.json`.

  See "About the Configuration Files".

- CDH Deployment

  Mammoth uses parcels instead of RPMs to deploy CDH.

- Apache Sentry

  Installation of Apache Sentry does not require `sentry-provider.ini` as a prerequisite.

Deprecated Features

The following features are deprecated in this release, and may be desupported in a future release:

- Mammoth Reconfiguration Utility

  The `bdacli` utility supersedes `mammoth-reconfig`. The `mammoth-reconfig` utility is only needed to change the disk encryption password.

  See "bdacli".

- MapReduce 1 (MRv1)

  YARN (MRv2) supersedes MRv1. Users who want to continue to use MRv1 on Oracle Big Data Appliance versions 3.x and 4.x should contact Oracle Support before using Mammoth to patch or upgrade the software.

- Disk Encryption

  A new encryption system that is more flexible and robust will replace the current system in an upcoming release.

## Changes in Oracle Big Data Appliance Release 4 (4.0)

The following are changes in Oracle Big Data Appliance release 4 (4.0):

New Features

- Oracle Big Data SQL 1.0.0

  Oracle Big Data SQL supports queries against vast amounts of big data stored in multiple data sources, including HDFS and Hive. You can view and analyze data from various data stores together, as if it were all stored in an Oracle database. Support for Oracle Big Data SQL includes the following new features in Oracle Database:

  - `DBMS_HADOOP` PL/SQL package

  - Hive static data dictionary views

  - Access drivers for Hadoop and Hive

  Oracle Big Data SQL is an installation option, which you can specify using the Oracle Big Data Appliance Configuration Generation Utility.

  You can monitor and manage Oracle Big Data SQL using the `bdacli` command and Cloudera Manager.

  See "bdacli" and Oracle Big Data Appliance Software User's Guide.

- Service Migration

  The `bdacli` utility can migrate services from a failing critical node to a healthy noncritical node. It can also remove failing critical and noncritical nodes from a cluster, and restore them to the cluster after repairs. See "bdacli" and Oracle Big Data Appliance Software User's Guide.

- Software Upgrades

  - Cloudera's Distribution including Apache Hadoop 5.1.0

  - Cloudera Manager 5.1.1

  - Perfect Balance 2.2.0

  - Oracle Data Integrator Agent 12.1.3.0 (for Oracle Big Data Connectors)

  See Oracle Big Data Appliance Software User's Guide.

- Oracle NoSQL Database Zone Support

  The Oracle Big Data Appliance Configuration Generation Utility and the mammoth -e command support multiple zones on Oracle NoSQL Database clusters. You can add nodes to an existing zone, or create a new primary or secondary zones.

  See "Oracle NoSQL Configuration" and "Mammoth Software Installation and Configuration Utility".

- Multiple Rack Clusters

  You can now install a cluster on multiple racks using one cluster_name-config.json file.